Papers with question inference
Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question? (2024.acl-long)
Copied to clipboard
| Challenge: | Multiple-choice question answering (MCQA) is often used to evaluate large language models . a recent study found that LLMs perform MCQA with choices-only prompts . |
| Approach: | They investigate whether LLMs can perform multiple-choice question answering (MCQA) with choices-only prompts . they find no evidence that the choices- only accuracy stems from memorization alone . |
| Outcome: | The results show that LLMs perform MCQA with choices-only prompts with 0.33 accuracy gain. |